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ABSTRACT 

We are so used to accept new technologies being the driver of change and innovation in human computer interfaces 
(HCI). In our research we focus on the development of innovations as a design process - or design, for short. We also 
refer to the entire process of creating innovations and putting them to use as “cognitive processes” - or cognition, for 
short. Scientific but simplistic design approaches often do not help to answer relevant questions in everyday computer 
design, hi two experiments we show how practical design questions can be examined: 1. “How does color and shape of a 
web site attract users and help to memorize its content?”2. “How does shape and texture influence the believability of 
computer generated characters?” The results show that design aspects such as color, shape, and texture greatly influence 
the emotional assessment and cognitive performance of users. 
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1. INTRODUCTION 

In interface design ‘form does not follow function’ (a function that is pre-defined by the programmer) rather 
than forms following users’ intentions and imaginations. Many designers will claim that “user centred 
design” and other usage oriented design processes already are built on this principle. Moreover, explicit 
psychological design models such as GOMS and SOAR are in use (cf. Rosenbloom, Laird & Newell, 1993). 
But the kind of ‘cognition’, which is modelled, and the kind of usability, which is enhanced, based on 
psychological usability guidelines is rather limited (e.g. Mayer, 2001). The debate about design “beyond 
usability” and “user experience” instead of simple usability reflects this insight (Sharp et al., 2007). 

Cognitive Design is often seen as a part of “Knowledge Media Design” (Stephan, 2006). This view 
clearly relies on the narrow view of cognition which sees cognitive design as a way to adapt visual design to 
human information processing through sensation, perception and logical thinking (Hasebrook, 1998; 
Hasebrook & Gremm, 1999). A broader view of intuitive cognition (Gigerenzer, 2007; Gigerenzer & Selten, 
2001) would include all mental abilities, such as imagination, intuition and affection. This does not aim at a 
purely affective or aesthetic design, although there is a clear overlap between cognitive design and aesthetic 
design - as there is an overlap between cognition and emotion (Eich et al., 2000). However, cognitive design 
aims at enabling cognitive processes, including imagination and intuition, by applying design processes to 
computer interfaces. Computer interface design is not so much about form (as for tangible products) but 
about users intentions. We may then define cognitive design as the formation and projection of users’ 
intentions onto functions provided by the software. 


2. STUDIES ON COGNTIVE DESIGN 

Decades of human computer interaction and user centered design practices has left us with millions of 
artifacts but little practical insight: We are hardly in the position to solve everyday design problem in a 
scientifically strict sense. Two question from our research illustrate that point: 
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• How does color help to design web sites in a way that both understanding and attractiveness are 
enhanced? And: 

• What design makes computer generated characters (in learning games and as avatar) attractive and 
believable? 

The reason for this shortage of applicable scientific knowledge is partly caused by the way (natural) 
sciences accumulate the ‘truth value’ (cf. Hofmann, 1999) and the still underdeveloped interdisciplinary 
research needed for successful human computer interaction (cf. Hasebrook & Saariluoma, 2007). Another 
reason is the way interface innovations are created: Most psychological design models, such as GOMS or 
SOAR, feature “design without mind”, because it shapes design only to automatic sensory information 
processing and selected parts of rational information processing - but not to the broader concept of cognition 
including intuition and imagination (Gigerenzer & Selten, 2001; Kahnemann, 2003). 

2.1 Study 1: Color, Emotion, and Memory 

Nicole Kohlrausch (2005) and I did an experiment using a simple but effective emotional assessment tool, 
called ‘Self Assessment Manikin’ (SAM; Bradley & Lang, 1994). 
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Figure 1. Self Assessment Manikin (SAM) introduced by Lang (1980) to assess users’ emotional status in a further 

develoed version by Suk (2006) 

Our study combined emotional self assessment and a cued recall test in an experimental within -subject 
design investigating the most relevant factors of the use of color in web design. The experimental design 
comprised the factors main color (yellow, blue, red, green), color spread (extensive vs. sparse) and color 
scheme (monochromatic vs. complementary). The designs were presented following welcome and instruction 
pages sin a fully counter -balanced Latin Square procedure, and at the end a brief memory test was 
administered (cf. figure 2). 

The sample consisted of 58 participants: 15 female, 42 male and 1 without specification. Most of the 
participants (78%) were in their twenties and thirties and professionals (71%), which together represents the 
portion of the population that accounts for most of the Internet users. The results of our study show that male 
and female subjects prefer different color sets and color schemes. All subjects rated designs with 
monochromatic and spare extension of color with a higher positive valence, less dominance and less arousal. 
However, a mild arousal and some dominance combined with a neutral valence proved to be the optimal 
design for (cued) recall in a memory test. 
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Figure 2. Overview of experimental procedure and designs to test the link between color, cognition and emotion in Web 

site design (Kohlrausch, 2005) 


2.2 Study 2: Animated Human and Non Human Characters 

The second question, mentioned above, was about believability and reality of computer generated characters, 
more specifically their facial expressions. Believability refers to human qualities, such as emotion and social 
behavior, whereas reality ratings assess the similarity of computer generated faces and real faces. The most 
cited model addressing this relationship of feelings about human qualities and the reality factor of simulated 
faces is the ‘uncanny valley’ hypotheses developed by Japanese robotics scientist, Masahiro Mori, in 1970. It 
states that a robot made more humanlike in its appearance and motions causes an increasingly positive 
emotional response until it reaches a point beyond which the response becomes repulsive. However, as the 
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appearance and motion becomes even less distinguishable from a human being, the emotional response 
becomes positive once more (Mori, 1970; MacDorman, 2006). 

We examined the emotional assessment of computer generated facial expressions with an experimental 
design comprising the factors ‘Form’ (human vs. non-human character), ‘Color’ (full color vs. greyscale), 
and ‘Texture’ (highly detailed or textured vs. low detail or texture). The experiment consisted of two parts: In 
the first part, we presented different film clips to the users displaying one clip after other. In the second part, 
both human and non-humans video clips were displayed side by side. In order to avoid sequence effects or 
systematic biases every single start of the survey produced a new random order of the video clips. Main 
effects and interactions of all factors on emotional ratings using the SAM tool as well as reality and 
believability ratings were collected. 


How do you evaluate the movie clip? 

joi] in 

* , -a 'o * 





Figure 3. Form to evaluate “reality” (like a real human face) and “believability” (show human behavior; left) and Self 
Assessment Manikin (see. figure 1), figures taken from Waqar-ul (2006) 

Forty-eight subjects responded to our online survey, 39 (81%) male and 9 (19%) female. Most of the 
participants are in their twenties and thirties 75%, (19% forties, 4% older). 50% participants come from 
media or arts background (13% education, 7% animation). Figure 2 (top) shows the impact of color and 
texture or human and non-human forms on valence. The figure on the left shows a significant interaction of 
the factors color and texture, that is, valence for colored human faces is higher when texture detail is high; for 
gray scale animations, however, valence ratings are higher when low textural detail is used. There is also a 
marginal significance of the three-way interaction of the factors color and texture with the form of the 
character (human vs. non-human). Figure 2 (middle) depicts the impact of color and texture on arousal. There 
are two main effects: The figures show a main effect of the factor texture: Arousal is higher for low texture 
animations. The form of the character causes the other main effect: Arousal is higher for non-humans than for 
human forms. Additionally, there is an interaction of the factor color and form, because color has an positive 
influence on arousal only if the face is human. Finally, figure 2 (bottom) also shows the impact of color and 
texture on dominance. There is a main effect of the form (human vs. non-human). There is also an interaction 
of the factors texture and color, that is, dominance is higher for high detail texture animations with color. The 
three-way interaction of the form, texture and color was significant, as well, showing that high detail 
produces high dominance ratings for human colored forms but not but for non-human forms or displays in 
gray scale. 
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Figure 4. Emotional assessment of valence (top), arousal (middle) and dominance (bottom) as a function of the factors 
humanity (human vs. non-human computer generated character), color (full color vs. grayscale), and texture (high 

textural detail vs. low textural detail) 

The results of the emotional ratings are supported by the results reported for reality and believability: The 
impact of color and texture in human and non-human forms on reality ratings shows a three-way interaction 
of form, color and texture, that is, reality ratings are high for colored human characters with high textural 
detail as well as gray scale human faces with low textural detail (non -humans show high reality ratings only 
for low textural detail). During the direct comparison task in the second part of the survey, there was a 
significant impact of factor human on reality assessments: Reality ratings are higher for human faces than for 
non-humans; believability ratings are higher, when high textural detail is used. 


3. CONCLUSION 

In conclusion, the simple decomposition of facial animations into the factors form, color and texture (cf. Lee 
& Magenat-Thalmann, 2000) as well as the decomposition of emotions into the factors valence, arousal and 
dominance (Bradley & Lang, 1994) gives detailed experimental support of the uncanny valley hypotheses 
and shows its possible cognitive and emotional components. Also, the same emotional assessment combined 
with a memory tests gives insight in the impact of color schemes into the emotional and rational processing 
of web pages and their content. Thus, it seems useful not only to include performance rating, such as speed 
and accuracy, and simple acceptance ratings (like or dislike) into design processes. Emotional assessment and 
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careful selection of holistic factors of experience, such as color and human faces, can substantially add to our 
design knowledge accumulated by technical and purely rational design processes. 

We used input from this research in order to design believable interactive interfaces to “knowledge 
robots”, or “knowbots”, for short (Hasebrook, 2008). Knowbots are an extension of a robot into a 
cyberspatial environment. The purpose of a knowbot is to search in cyberspace for a desired informational 
resources. As people are not able to cope with the exponential growth of information and the increasing 
speed of information interchange, intelligent technical support information retrieval and selection is needed. 
The ability and likeliness to use such technical support is depending on the expected usefulness and easiness 
of the interface. Visual knowbots differ from other interface metaphor of search engines by immersion and 
the believable use of communicational elements, such as simple question and answer dialogues (cf. figure 5). 



Figure 5. A moving and talking “Knowbot” as communicative interface to a virtual landscape (Hasebrook, Erasmus & 

Doeben-Henisch, 2001) 


If we accept the notion of user experience as a relevant part of design, the design process and the 
evaluation of design outcomes become a social process. With the advent of the social “Web 2.0” the ecology 
of Web information has changed (Huberman, 2001) driving not only disruptive technical change but also 
social change. New skills, abilities and a combination of competences is needed in order to scope with social 
networks and continuously updated information bases. The call for more interdisciplinary scientific work 
cannot be the major force driving interface innovations, simply because a new paradigm will focus on further 
development of psychological theories and models (cf. Hasebrook & Saariluoma, 2007). We believe that new 
interfaces will derive from user experience and the social process this creates. Vytgotsky (1987) described 
this the mediating function of media: Every user is a part of a social mind, called culture, an individual mind, 
generating conscious experiences, and a human body (or embedded mind), the model which more and more 
drives advances in robotics. 
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